Metaheuristic Agent Processes (MAPs)

characterizations of ABMs, however, may be viewed chiefly as “after-the-fact” attempts to group together ideas that are intuitively conveyed by the agent terminology. While a thoroughly precise and universally agreed-upon definition of agent-based models may not exist, the relevance of ABMs in science and industry is manifested in its diverse applications. These include explorations into the transmission of diseases, the operation of ecosystems, the dynamics of supply chains, the fluctuations and trends of financial markets, the behavior of economic sectors, the patterns of migrations, the flows of metropolitan traffic, the interactions of chemical and bio-physical processes and so forth. Within these numerous and varied systems, two essential elements come conspicuously to the fore: the need for complex simulations and the need for highly adaptive optimization. The relevance of complex simulations has been long been recognized, and has received extensive attention in agent-based modeling – as evidenced by the existence of public domain libraries for 1 Complexity in this case is manifested in the outcomes of the simulation, though not necessarily in the elements and operations that compose it. Kyoto, Japan, August 25–28, 2003 P2-2 MIC2003: The Fifth Metaheuristics International Conference generating simulations from an agent-based perspective. On the other hand, the relevance of optimization has been significantly underplayed. No doubt this is because the structure of many agent-based systems cannot easily be captured by classical optimization models, due to conditions of non-linearity, discreteness and/or uncertainty that are often present in these systems. In fact, the current role of optimization in agent-based modeling is entirely analogous to the role it assumed within the general field of simulation only a few years ago, as a result of these same factors – inapplicability of classical models, non-linearity, combinatorial complexity and uncertainty. Within the simulation industry, practitioners struggled along for years in an attempt to handle the compelling issues of optimization simply by means of trial-and-error analysis. The simulation literature would often claim to have “optimized” various system components, based on nothing more than a series of guesses and brute force re-trials. Today, this picture has dramatically changed, thanks to the newly-emerged metaheuristic procedures that are now routinely being used in the simulation industry, and that are creating solutions of vastly greater quality and utility than were previously possible. In the same way, optimization within agent-based models is approached for the most part by resorting to a series of educated guesses about the values of various input control parameters and decision variables. There is no globally coordinated mechanism for identifying parameter values that yield outcomes of high quality. In particular, the possibility of conducting an intelligent search for high quality solutions by using an appropriate metaheuristic framework is still largely unrecognized. 2 Metaheuristic Agent Processes The purpose of this paper is to underscore the opportunity that exists to expand the scope and power of agent-based models by integrating them with metaheuristics to create Metaheuristic Agent Processes (MAPs). From a strictly technical point of view, this step involves nothing revolutionary, since it corresponds to the same type of advance already made in the realm of simulation. Such a development is all the more natural because of the close alliance between simulation and agent-based models, whereby simulation is pervasively used to capture the dynamics and investigate the implications of many forms of ABMs. From a practical point of view, the creation of metaheuristic agent processes to improve the quality and value of 2 A prominent example is the Swarm Simulation System (www.swarm.org). An award-winning commercial authoring tool for creating agent-based models is provided by AgentSheets (www.agentsheets.com). 3 The leading provider of this technology to the simulation industry is OptTek Systems (www.opttek.com). Kyoto, Japan, August 25–28, 2003 MIC2003: The Fifth Metaheuristics International Conference P2-3 information derived from agent-based models would mark a significant step forward. The integration required to produce effective MAPs rests on principles already well-known and applied within many segments of the metaheuristic community. Indeed, a few metaheuristic procedures are founded on metaphors that call to mind the notions and terminology of agent-based systems, and some proponents of these metaheuristics have already sought to have their work viewed as a contribution to the ABM area. However, such contributions are still limited in scope, and contributions of a more substantial nature are not only possible but greatly needed. The opportunity to make gains by the creation of MAPs rests on the same types of metaheuristic advances that have made it possible to handle the complex conditions of non-linearity, discreteness and uncertainty in other realms. The next sections set out to accomplish three things. First, we demonstrate an intimate connection whereby certain long-standing metaheuristic strategies may be viewed as instances of agent-based processes themselves. Thus, from this standpoint, there are compelling precedents for a broader integration of metaheuristics and agent-based models to produce MAPs. Second, within this development we also identify recent innovations that hold promise for further enriching the realm of metaheuristic agent processes. Finally, we discuss the opportunity for next steps that can usefully expand the application of agent-based models by their integration with metaheuristics. 3 Metaheuristic Processes Conceived as Agent-Based Systems We illustrate a few selected metaheuristic strategies that have conspicuous interpretations as agent-based systems, although the first two of these strategies we discuss emerged long before the notions of agent-based models were popularized. On the basis of these interpretations, it will be clear that many other metaheuristic strategies likewise can be viewed as instances of an agent-based framework. Thus, while agent-based modeling and optimization have up to now remained somewhat insulated from each other, the two fields can productively be viewed as interrelated through the design of metaheuristics and, in particular, through the realm of MAPs. We begin by describing strategies designed to generate solutions within the setting of job shop scheduling by creating improved local decision rules. The first of these approaches (Crowston, et al., 1963) seeks to create improved rules by selecting probabilistically from a collection of 4 These metaheuristics are grouped by the label of “swarm intelligence” or “particle swarm optimization,” and widely portrayed by the metaphor of bees swarming about a hive. Interesting and perhaps unexpected bonds to certain other types of methods are evidenced by the fact that this search metaphor was originally introduced in the literature of tabu search and adaptive memory programming (see, e.g., Glover (1996) and Glover and Laguna (1997)). A website on particle swarm optimization can be found at www.particleswarm.com. Kyoto, Japan, August 25–28, 2003 P2-4 MIC2003: The Fifth Metaheuristics International Conference known rules so that different rules will be applied at different decision points throughout the process of generating a schedule constructively. As complete solutions (schedules) are produced, the decision rules that are more often used to create the solutions of higher quality receive greater increases in the probabilities for applying them in future passes. The process can be viewed from as a metaheuristic agent process as shown in Fig. 1. The decision rules (of which there may be many) operate as agents, and at each step of constructing a solution the agents enter into a “probabilistic competition” to determine which rule is allowed to augment the current solution to create an expanded solution for the next stage. The process repeats until completing the generation a new solution, whereupon the updated probabilities are calculated and the procedure begins once more with the null solution, to launch another construction. (For simplicity, we do not try to show all connections in this or subsequent figures, or to identify stopping rules, as typically based on numbers of iterations and/or quality of solutions obtained.) It is to be noted that this type of approach can readily be applied in many other settings, as a multi-start metaheuristic. Also, in the case where different decision rules are used to choose among alternative neighborhoods, the approach can be envisioned as an instance of a probabilistic form of strategic oscillation (Glover and Laguna, 1997; Gendreau, 2003) or as a variable neighborhood search procedure (Hansen and Mladenovic, 2003). A related, but somewhat more effective method (Glover, 1963) replaces the approach of probabilistically choosing among a basic collection of rules by instead creating new rules that Kyoto, Japan, August 25–28, 2003 MIC2003: The Fifth Metaheuristics International Conference P2-5 are explicitly different from all members of the collection, using a process of parametric combination. The basic rules are first re-expressed to yield an evaluation metric compatible with the notion of creating a weighted combination, and then each new pass systematically modifies the weights used to combine rules on preceding passes. The design of this approach later became encapsulated in surrogate constraint methods, by combining constraints instead of decision rules, and also more recently embodied in scatter search procedures (see the surveys respectively of Glover (2003) and Glover, Laguna and Marti (2000)). From the perspective of a metaheuristic agent process, the rules to be combined may again be viewed as the agents. The diagrammatic outline in Fig. 2 also refers to the more general form of the process that includes surrogate constraint and evolutionary scatter search approaches, by allowing constraints and solutions to be agents instead of decision rules. In this type of process we may conceive of an additional agent entering the picture (a “marriage agent”) as the means for creating the weighted combination of the components. Another generalization operates here, because the procedure is not only concerned with augmenting partial solutions (in successive stages of construction), but also with transforming complete solutions directly into other complete solutions. The augmentation repeats until creating a complete solution, while the direct transformation creates a new solution at each step. At this point, the new weights are produced for the next pass, and the procedure iterates until satisfying a chosen stopping criterion. An instance of this approach called a “ghost image process” has produced the best known solutions for an extensive test-bed of fixed charge transportation problems (Amini, 2003). There are evidently a variety of possible variations that likewise fit within the same agent-based design, such as permitting different weights to be applied at different stages of construction or Kyoto, Japan, August 25–28, 2003 P2-6 MIC2003: The Fifth Metaheuristics International Conference according to different environments. Likewise, as remarked earlier, the approach can be used as a schematic for a multi-start method and decision rules can refer to rules for choosing neighborhoods. Finally, we observe that the approaches of Fig. 1 and Fig. 2 can be merged, to create a probabilistic variation of a parametric combination process. Within the framework of evolutionary approaches, we may consider the extension of scatter search procedures represented by path relinking methods (see, e.g., Glover, Laguna and Marti (2000) and Yagiura, Ibaraki and Glover (2002)). Path relinking combines solutions by generating paths in neighborhood spaces instead of by generating paths in Euclidean spaces (as occurs in scatter search). To provide fuller generality, we treat rules and solutions alike as agents, thereby encompassing path relinking methods that use both transitional and constructive neighborhoods. The same representation, under appropriate qualification, can also capture the approach of referent domain optimization (Glover and Laguna (1997) and Mautor and Michelon (1998)). The depiction of these approaches as metaheuristic agent processes is given in Fig. 3. Path relinking using transitional neighborhoods is embodied in this diagram by focusing on solutions as agents. At each step, a subset selection process is applied (which can also be viewed as performed by an agent), and the solutions in the subset are joined by moving from selected initiating solutions through neighborhood space toward other members of the subset, which provide guiding solutions for determining the trajectory. New solutions are culled from this 5 Transitional neighborhoods define moves that transform complete solutions into other complete solutions, while constructive neighborhoods define moves that transform incomplete (partial) solutions into more nearly complete solutions. Kyoto, Japan, August 25–28, 2003 MIC2003: The Fifth Metaheuristics International Conference P2-7 procedure by an intermediate selection step and subjected to an improvement process. Finally, an evaluation filter decides which of the resulting solutions enters the set of agents to be considered for the next round, by replacing previous agents that are dominated according to intensification and diversification criteria. Applications of this approach are surveyed in Glover, Laguna and Marti (2000, 2003). A slightly altered focus where rules also take the role of agents occurs in a form of path relinking involving the use of constructive neighborhoods. In this case, solutions and rules are intermingled, by a design where the guiding solutions provide “votes” (i.e., components of an overall evaluation) that determine which solution element is the next in sequence to be added by the constructive process. This type of approach has recently been applied effectively in the context of satisfiability problems by Hao, Lardeux and Saubion (2003), yielding new best solutions to a number of benchmark problems. A variant applied in conjunction with ejection chain neighborhoods by Yagiura et al. (2002) succeeds in generating solutions for generalized assignment problems that are not matched by any other procedure. Referent Domain Optimization is captured by this diagram under the condition where the new solutions produced by the solution combination mechanism are subdivided into components (domains), and the improvement process tightly constrains some of these components while subjecting the remaining problem to an intensified improvement procedure, typically an exact solution method. Our final illustration of a metaheuristic from the standpoint of an agent-based process concerns the Filter-and-Fan procedure (Glover, 1998; Greistorfer, Rego and Alidaee, 2003). This approach operates with solutions as agents to produce the MAP representation shown in Fig. 4, which portrays a single iterative stage of the procedure. Each iteration begins by performing a fan step whereby each agent (solution) generates a subset of descendants in its neighborhood. The solutions carry individual memories (of the type customarily used in tabu search) which are used to separate and remove certain descendants (Filter A). The system as a whole also carries an associated global memory that removes additional descendants (Filter B). Finally, evaluation criteria based on intensification and diversification eliminate certain remaining solutions (Filter C), and those solutions that survive the gauntlet generate an updated solution pool for re-applying the process. After a chosen number of steps, the method recovers a restricted subset of solutions from earlier steps that resulted in the best solution(s) in the final pool. The recovered solutions then compose the starting solution pool for the next stage of the process. 6 As previously noted, not all connections are shown, in order to keep the diagrams from being cluttered. 7 Destructive as well as constructive neighborhoods are typically incorporated in such designs. 8 Referent domain optimization has several forms, but we limit consideration to this straightforward version for the purpose of illustrating an agent-based representation. Kyoto, Japan, August 25–28, 2003 P2-8 MIC2003: The Fifth Metaheuristics International Conference The investigation of Greistorfer, Rego and Alidaee (2003) discloses that this approach dominates all previous metaheuristic procedures for solving facility location problems . 4 General Observations on MAP Representations The representations of metaheuristics as MAPs illustrated in the preceding section are intended to be suggestive rather than exhaustive. Evidently, their form can be adapted to create agent-based representations of a variety of other metaheuristic approaches. For example, we may conceive associated constructions for representing processes that might be called Neural MAPs, Genetic MAPs, Evolutionary MAPs, Ant Colony MAPs, Variable Neighborhood MAPs, and so forth. The depiction of metaheuristics as MAPs has the benefit of clarifying the connection that already exists between metaheuristic procedures and the realm of agent-based models. On the other hand, these representations also suffer a significant limitation, by embodying a level of abstraction that loses track of details that are critical for producing the most effective methods. Among a wide range of procedures that might be portrayed within the same representational framework, only a small subset will reach or surmount the level of performance achieved by the methods that have motivated these MAP diagrams. The foregoing representations are also incomplete in another respect, resulting from their restricted focus on metaheuristics in isolation from other types of agent-based processes. The methods produced by this restricted focus might be called “antecedent MAPs,” or α-MAPs, to Kyoto, Japan, August 25–28, 2003 MIC2003: The Fifth Metaheuristics International Conference P2-9 distinguish them from the more ambitious MAPs that integrate metaheuristics with agent-based models of other forms. It is worth re-emphasizing that this integration is essential to accomplish the goal of bringing optimization to bear on ABMs. Notably, an important step toward creating fully integrated MAPs has been effected by the creation of a library of functions, called the OptQuest Engine, that integrates optimization with simulations that have an entirely general form. The library is not limited to serving as a tool for academic research, but has been widely used in practical applications and has been adopted by nearly all major providers of commercial simulation software. Consequently, it is a natural next step to structure such metaheuristic processes to handle the specific manifestations of simulation that occur in agent-based modeling. Adaptations of this form can be tailored to yield integrated MAPs that exploit the features of different classes of applications, thereby increasing their usefulness. The resulting higher-order MAPs afford the opportunity to significantly extend the capabilities of current agent-based methods, by making it possible to deal more effectively and comprehensively with environments attended by uncertain events, and to perform improved analyses of systems involving agents that behave according to probabilistically defined parameters. The unmatched ability of metaheuristics to handle complex nonlinearities and combinatorial relationships provides additional motivation for creating MAPs that go beyond the current “grope-in-the-dark” applications of what-if analysis, and that open the door to the benefits of advanced optimization capabilities for agent-based modeling.